Skip to content

Feat 698 equity points - #714

Merged
eaitbrahim merged 3 commits into
mainfrom
feat-698-equity-points
Sep 4, 2026
Merged

Feat 698 equity points#714
eaitbrahim merged 3 commits into
mainfrom
feat-698-equity-points

Conversation

@eaitbrahim

Copy link
Copy Markdown
Contributor

What & why

Tests-first evidence

  • Tests written first, seen failing for the right reason

Gates (all must pass)

  • uv run ruff check clean
  • uv run mypy clean
  • uv run pytest -q green

Scope check

  • This PR touches a rail or a default classification — checked means it DOES;
    leave checked only if true, and if so: cite the source and open the discussion
    BEFORE review (CONTRIBUTING.md, "Governance: rulings vs. machinery").
  • New dependency added (needs discussion first)

eaitbrahim and others added 3 commits September 4, 2026 04:12
#698)

`agent._mark_to_market_equity` has always run once per cycle, and nothing kept the
answer. What survived was `agent_state["equity_history"]`: a 7-day window that exists
for the weekly drawdown rail and that `record_external_flow` REWRITES on a declared
deposit. That is a rail's working set, not a record -- so the series is written
alongside it rather than derived from it, and neither reconstructs the other.

`equity_points` (schema v19) is the record. Per row: the reading, the mode that
produced it, and the high-water mark in force AFTER it was folded in -- read back by
the chart rather than recomputed, because a rebased HWM is not the running maximum of
the equity and the overlay's whole point is the ceiling rail 11 actually had.

`mode` is the load-bearing partition, taken from the same `equity_state_mode` stamp
`_clear_live_mode_if_needed` reads before wiping the shared HWM on a flip. Deriving it
a second way at the call site would let two answers to one question drift, and the
failure is silent: a mislabelled row does not go missing, it lands in the other
account's curve. An UNSTAMPED mode writes nothing and never guesses -- and never
raises, so the chart can never take rail 11 down with it.

`cash` and `unrealized` are the split, and they are passed, never derived: this
function has a total and no positions. NULL means "not recorded", never zero -- the
`orders.filled_quantity` convention. Both agent branches now supply them from ONE pass
over ONE set of balances and marks (`_mark_to_market_parts`, `PaperTrader.unrealized`),
so a price cannot tick between the total and its parts and file a row that does not
reconcile. `unrealized_on_marks` mirrors `mark_positions` guard for guard: a position
valued at cost contributes zero, because the equity in the same row valued it at cost
too. Tested as an identity: cash + cost basis + unrealized == equity.

NO BACKFILL, and what could be backfilled is exactly what must not be. Replaying the
shifted 7-day window would publish equities the account never had; the orders ledger
would give a closed-trade curve with no cash leg, a different quantity wearing this
table's name. An empty table means "not observed before v19", which is true.

`build_equity_series` is a NEW builder, not a change to `build_equity_curve`. That one
plots cumulative net P&L over closed trades and argues for its trade-order axis on its
own terms -- correctly, for that quantity. This plots what the account was worth
whether or not it traded, where the gaps between cycles are the information. Segmented
by RUNS of mode (paper, live, paper is three), bounds spanning the overlays so a rail
floor cannot fall off the box, and zero deliberately NOT forced onto the axis: an
account is not up or down against nothing.

Three departures from the issue text:

- v19, not v16: main reached 18 after the issue was filed.
- `ts INTEGER`, not TEXT: every timestamp in this schema is an epoch int, and TEXT
  orders epochs lexically -- the `ts >= ?` window the chart reads by would silently
  return the wrong rows across a digit-count change. Pinned by a test.
- the record is `EquityReading`: `keel.commands.insights` already owns an
  `EquityPoint`, a PLOT point, and the chart module now uses both.

Still to come on #698: the payload/template/chart.js wiring that puts the series on
the Insights page.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…curve (#698)

Portfolio reality above statistical expectancy. `equityChart` keeps plotting cumulative
net P&L over closed TRADES on a trade-order axis -- `build_equity_curve` argues for that
axis on its own terms, and the argument is still right for that quantity. Above it now
sits what the ACCOUNT was worth every cycle, traded or not, on a time axis, where the
quiet week is the information rather than noise. Two questions, two charts; neither
replaces the other.

payload.py -- `equity_series_payload`, and `insights_payload` gains a REQUIRED `series=`
keyword for the reason `journal_payload`'s `curve=` is one: a default would quietly
serve a view with no chart. Segments cross AS SEGMENTS, never as one flat list; the
mode partition is the whole reason the shape exists, and flattening is the single
operation this wire must not make easy. Coordinates are bare strings, everything a
reader is told is a `Field`, and `dd_floor_y` is `null` rather than `"0"` when the rail
setting is unknown -- zero is the TOP of an SVG box, so a `"0"` would draw a ceiling in
force above every reading.

The reading -- the sentence a reader who cannot see the chart is given -- names the
modes, because that is the part a spoken summary would otherwise flatten into "an
account went from ten thousand dollars to two hundred and fifty". It says the lines are
separate accounts ONLY when there is a partition to explain: told to a paper-only
deployment it sends a reader looking for a line that is not there. `state` is neutral,
unlike the curve's: a curve closes on a P&L, which is a verdict, and a series closes on
a balance, which is not one.

`modes` and `is_partitioned` are properties of `EquitySeries`, not a list the serialiser
assembles. Rule 6e bans `len()` in payload.py so that a count on the wire is one the
report already holds, and the first draft of that sentence broke it -- the rule was
written down in `EquityCurve.point_count`'s own docstring the whole time.

api.py -- the series is read on `/api/insights`, not on `/api/journal` beside the curve
it is drawn above, because it describes the ACCOUNT and not the closed trades. The
journal's curve narrows with that endpoint's `?limit=`; this narrows with nothing, and
two charts that answer a query differently must not share a payload where a reader would
assume they agree. `max_total_dd_pct` comes off the same loaded config
`build_insights_report` reads, so the floor under the chart and the ceiling quoted in
the account card beside it are one setting rather than two reads that can drift. Every
recorded cycle, never a default window: cutting it to a span here would reintroduce the
exact horizon this issue exists to remove.

chart.js -- `equitySeriesChart`: one polyline per segment, plus the high-water mark and
the drawdown floor drawn beneath the account line in the token the baseline already
uses. The canvas is `svg.series` and NOT `svg.curve` on purpose: #602's wheel-zoom,
drag-to-pan and cursor legend reach for `svg.curve` with `querySelector`, which takes
the first match, and this chart renders first. Paper is dashed and live is solid, so the
two accounts stay distinguishable in greyscale, on e-ink and with red-green colour
deficiency -- the same rule the losing-segment dash already follows, and the same
principle as payload.py's ▲/▼.

render.js -- stacked, with a note saying the top chart covers every recorded cycle
rather than the journal's row cap, since two charts side by side otherwise invite the
assumption that the cap applies to both. A payload with no `equity_series` at all draws
nothing and says nothing: that is not a deployment with no readings, it is a response
from before the field existed, which a service worker can still be holding after an
upgrade -- and reading `.reading.display` off it would blank the whole view over a stale
cache entry that the next refresh fixes on its own.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…able unbounded (#698)

Two review findings on the equity series, plus the test that was missing.

THE REGRESSION. `equitySeriesChart` set `figure.className = "chart"`, and `main.js`
reaches for the chart's WRAPPER twice -- `highlightJournalRow` and the chart-action
handler -- with `contentNode.querySelector("figure.chart")`, which takes the first match
in document order. The series is appended ABOVE the curve, so both lookups landed on it;
`highlightTrade` then found no `.highlight` group, returned early, and hovering a journal
row on /insights silently highlighted nothing. It worked before this branch, so it was a
pure regression.

The original reasoning got the CANVAS right and stopped there: `svg.series` was chosen
deliberately so the #602 pointer gestures could not retarget, and the comment says so at
length. It never asked what else `main.js` selects on. The figure is now `"chart series"`
and both lookups carry `:not(.series)`; a test pins both halves, because either alone
leaves the collision live.

THE UNBOUNDED READ. `/api/insights` read the whole `equity_points` table on every
request. The table is append-only and grows one row per cycle forever -- at the default
`auto_trade.interval_sec` of 900 that is ~35,000 rows a year -- and `main.js` re-polls
the view every 15 seconds, rebuilding one SVG circle per point each time. Measured at
580 bytes per point in the rendered payload, a year of the hourly paper profile is
4.8 MiB per response, re-fetched four times a minute. The previous note called the
unbounded read deliberate ("every recorded cycle, never a default window"); the choice
was defensible and its cost was not bounded, which is the half that did not hold.

`EQUITY_POINT_LIMIT = 1000`, newest first, sized off `PLOT_WIDTH`: past one reading per
plot unit the extra rows land on coordinates already drawn, so they cost bytes with
nothing on screen to show for it. A fixed cap rather than a `?limit=` because this route
carries no collection to page and the whole series is on disk for anyone who wants it.

A BOUNDED READ HAS TO SAY SO. Silently starting the line wherever the cap fell would
misstate the span of the record while every individual point stayed true -- the harder
kind of wrong to notice, and the kind this chart exists to avoid. So
`count_equity_points()` rides along, `EquitySeries` carries `total_recorded` and
`is_truncated`, the spoken reading says "the most recent N" and names the total, and the
note under the chart reads "N of M recorded cycle(s)". `total_recorded=None` means the
caller did not say, which is deliberately NOT a claim of completeness: inventing
"complete" from silence is the failure that guard exists for.

`get_equity_points(limit=)` keeps the most recent N and still returns them oldest first
-- the ordering is the caller's contract, not an artefact of how rows were selected -- and
the limit applies inside the mode filter, so asking for the last two live readings on a
paper-heavy database cannot return nothing.

THE MISSING TEST. All three agent tests were on the live branch. The paper branch now has
one through the real loop, asserting the recorded parts add back to the recorded total on
a cycle whose account seeded and traded before the reading -- the case where a total and a
split could come from two different states of the account. Both halves of the wiring were
mutation-checked.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@eaitbrahim
eaitbrahim merged commit 4616e10 into main Sep 4, 2026
4 checks passed
@eaitbrahim
eaitbrahim deleted the feat-698-equity-points branch September 4, 2026 11:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant